Cooperative human robot interaction systems: IV. Communication of shared plans with Naïve humans using gaze and speech

Cooperation is at the core of human social life. In this context, two major challenges face research on human-robot interaction: The first is to understand the underlying structure of cooperation, and the second is to build, based on this understanding, artificial agents that can successfully and safely interact with humans. Here we take a psychologically grounded and human-centered approach that addresses these two challenges. We test the hypothesis that optimal cooperation between a naïve human and a robot requires that the robot can acquire and execute a joint plan, and that it communicates this joint plan through ecologically valid modalities including spoken language, gesture and gaze. We developed a cognitive system that comprises the human-like control of social actions, the ability to acquire and express shared plans and a language and speech synthesis stage. This cognitive system was driving the actions of an iCub humanoid robot that maintained dyadic interactions with a single human actor. In order to test the psychological validity of our approach we tested 12 naïve subjects in a cooperative task with the robot. We experimentally manipulated the presence of a joint plan (vs. an individual or solo plan), the use of task-oriented gaze and gestures, and the use of language accompanying the unfolding plan. The quality of cooperation was analyzed in terms of proper turn taking, collisions and cognitive errors. Results showed that while successful turn taking could take place in the absence of the explicit use of a joint plan and/or its accompanying cues, the presence of a joint plan yielded significantly greater success. One advantage of the solo plan was that the robot would always be ready to generate actions, and could thus adapt if the human intervened at the wrong time, whereas in the joint plan the robot expected the human to take his/her turn. Interestingly, when the robot represented the action as involving a joint plan, gaze provided a highly potent nonverbal cue that facilitated successful collaboration and reduced errors in the absence of verbal communication. These results support the cooperative stance in human social cognition, and suggest that cooperative robots should employ joint plans, fully communicate them in order to sustain effective collaboration while being ready to adapt if the human makes a midstream mistake. Stéphane Lallée and Paul Verschure are with SPECS, UPF & ICREA Barcelona, Spain, Katharina Hamann & Jasmin Steinwender are with the Max Planck Institute EVA Leipzig. Felix Warneken is with the Dept. Psychology, Harvard University. Uriel Martienz & Hector Barron-Gonzales are with the University of Sheffield, Ugo Pattacini, Illaria Gori & Giorgio Metta are with the Italian Institute of Technology, Maxime Petit & Peter Ford Dominey are with the Robot Cognition Laboratory, INSERM, Lyon. [email protected] 1